Regularized Principal Component Analysis for Spatial Data

In many atmospheric and earth sciences, it is of interest to identify dominant spatial patterns of variation based on data observed at p locations and n time points with the possibility that p > n. While principal component analysis (PCA) is commonly applied to find the dominant patterns, the eigenimages produced from PCA may exhibit patterns that are too noisy to be physically meaningful when p is large relative to n. To obtain more precise estimates of eigenimages, we propose a regularization approach incorporating smoothness and sparseness of eigenimages, while accounting for their orthogonality. Our method allows data taken at irregularly spaced or sparse locations. In addition, the resulting optimization problem can be solved using the alternating direction method of multipliers, which is easy to implement, and applicable to a large spatial dataset. Furthermore, the estimated eigenfunctions provide a natural basis for representing the underlying spatial process in a spatial random-effects model, from which spatial covariance function estimation and spatial prediction can be efficiently performed using a regularized fixed-rank kriging method. Finally, the effectiveness of the proposed method is demonstrated by several numerical examples.     

One-Step Estimator Paths for Concave Regularization

The statistics literature of the past 15 years has established many favorable properties for sparse diminishing-bias regularization: techniques that can roughly be understood as providing estimation under penalty functions spanning the range of concavity between ?₀ and ?₁ norms. However, lasso ?₁-regularized estimation remains the standard tool for industrial Big Data applications because of its minimal computational cost and the presence of easy-to-apply rules for penalty selection. In response, this article proposes a simple new algorithm framework that requires no more computation than a lasso path: the path of one-step estimators (POSE) does ?₁ penalized regression estimation on a grid of decreasing penalties, but adapts coefficient-specific weights to decrease as a function of the coefficient estimated in the previous path step. This provides sparse diminishing-bias regularization at no extra cost over the fastest lasso algorithms. Moreover, our gamma lasso implementation of POSE is accompanied by a reliable heuristic for the fit degrees of freedom, so that standard information criteria can be applied in penalty selection. We also provide novel results on the distance between weighted-?₁ and ?₀ penalized predictors; this allows us to build intuition about POSE and other diminishing-bias regularization schemes. The methods and results are illustrated in extensive simulations and in application of logistic regression to evaluating the performance of hockey players. Supplementary materials for this article are available online.     

Variational Bayes With Intractable Likelihood

Variational Bayes (VB) is rapidly becoming a popular tool for Bayesian inference in statistical modeling. However, the existing VB algorithms are restricted to cases where the likelihood is tractable, which precludes their use in many interesting situations such as in state--space models and in approximate Bayesian computation (ABC), where application of VB methods was previously impossible. This article extends the scope of application of VB to cases where the likelihood is intractable, but can be estimated unbiasedly. The proposed VB method therefore makes it possible to carry out Bayesian inference in many statistical applications, including state--space models and ABC. The method is generic in the sense that it can be applied to almost all statistical models without requiring too much model-based derivation, which is a drawback of many existing VB algorithms. We also show how the proposed method can be used to obtain highly accurate VB approximations of marginal posterior distributions. Supplementary material for this article is available online.     

Precision Matrix Estimation With ROPE

It is known that the accuracy of the maximum likelihood-based covariance and precision matrix estimates can be improved by penalized log-likelihood estimation. In this article, we propose a ridge-type operator for the precision matrix estimation, ROPE for short, to maximize a penalized likelihood function where the Frobenius norm is used as the penalty function. We show that there is an explicit closed form representation of a shrinkage estimator for the precision matrix when using a penalized log-likelihood, which is analogous to ridge regression in a regression context. The performance of the proposed method is illustrated by a simulation study and real data applications. Computer code used in the example analyses as well as other supplementary materials for this article are available online.     

A Cross-Entropy Approach to the Estimation of Generalized Linear Multilevel Models

In this article, we use the cross-entropy method for noisy optimization for fitting generalized linear multilevel models through maximum likelihood. We propose specifications of the instrumental distributions for positive and bounded parameters that improve the computational performance. We also introduce a new stopping criterion, which has the advantage of being problem-independent. In a second step we find, by means of extensive Monte Carlo experiments, the most suitable values of the input parameters of the algorithm. Finally, we compare the method to the benchmark estimation technique based on numerical integration. The cross-entropy approach turns out to be preferable from both the statistical and the computational point of view. In the last part of the article, the method is used to model the probability of firm exits in the healthcare industry in Italy. Supplemental materials are available online.     

Semiparametric Bayesian inference for mean-covariance regression models

In this paper, we propose a Bayesian semiparametric mean-covariance regression model with known covariance structures. A mixture model is used to describe the potential non-normal distribution of the regression errors. Moreover, an empirical likelihood adjusted mixture of Dirichlet process model is constructed to produce distributions with given mean and variance constraints. We illustrate through simulation studies that the proposed method provides better estimations in some non-normal cases. We also demonstrate the implementation of our method by analyzing the data set from a sleep deprivation study.     

Some closed form robust moment‐based estimators for the MEM(1,1)

In this paper, we extend the closed form moment estimator (ordinary MCFE) for the autoregressive conditional duration model given by Lu et al (2016) and propose some closed form robust moment‐based estimators for the multiplicative error model to deal with the additive and innovational outliers. The robustification of the closed form estimator is done by replacing the sample mean and sample autocorrelation with some robust estimators. These estimators are more robust than the quasi‐maximum likelihood estimator (QMLE) often used to estimate this model, and they are easy to implement and do not require the use of any numerical optimization procedure and the choice of initial value. The performance of our proposal in estimating the parameters and forecasting conditional mean μ_t of the MEM(1,1) process is compared with the proposals existing in the literature via Monte Carlo experiments, and the results of these experiments show that our proposal outperforms the ordinary MCFE, QMLE, and least absolute deviation estimator in the presence of outliers in general. Finally, we fit the price durations of IBM stock with the robust closed form estimators and the benchmarks and analyze their performances in estimating model parameters and forecasting the irregularly spaced intraday Value at Risk.     

An Extremal Problem Arising in the Dynamics of Two-Phase Materials That Directly Reveals Information about the Internal Geometry

In two-phase materials, each phase having a non-local response in time, it has been found that for some driving fields the response somehow untangles at specific times, and allows one to directly infer useful information about the geometry of the material, such as the volume fractions of the phases. Motivated by this, and to obtain an algorithm for designing appropriate driving fields, we find approximate, measure independent, linear relations between the values that Markov functions take at a given set of possibly complex points, not belonging to the interval [-1,1] where the measure is supported. The problem is reduced to simply one of polynomial approximation of a given function on the interval [-1,1] and, to simplify the analysis, Chebyshev approximation is used. This allows one to obtain explicit estimates of the error of the approximation, in terms of the number of points and the minimum distance of the points to the interval [-1,1]. Assuming this minimum distance is bounded below by a number greater than 1/2, the error converges exponentially to zero as the number of points is increased. Approximate linear relations are also obtained that incorporate a set of moments of the measure. In the context of the motivating problem, the analysis also yields bounds on the response at any particular time for any driving field, and allows one to estimate the response at a given frequency using an appropriately designed driving field that effectively is turned on only for a fixed interval of time. The approximation extends directly to Markov-type functions with a positive semidefinite operator valued measure, and this has applications to determining the shape of an inclusion in a body from boundary flux measurements at a specific time, when the time-dependent boundary potentials are suitably tailored. © 2022 Wiley Periodicals, Inc.     

Estimation of IIR Systems with Binary-Valued Observations*

Estimation and control problems with binary-valued observations exist widely in practical systems. However, most of the related works are devoted to finite impulse response (FIR for short) systems, and the theoretical problem of infinite impulse response (IIR for short) systems has been less explored. To study the estimation problems of IIR systems with binary-valued observations, the authors introduce a projected recursive estimation algorithm and analyse its global convergence properties, by using the stochastic Lyapunov function methods and the limit theory on double array martingales. It is shown that the estimation algorithm has similar convergence results as those for FIR systems under a weakest possible non-persistent excitation condition. Moreover, the upper bound for the accumulated regret of adaptive prediction is also established without resorting to any excitation condition.